Enhancements in display resolution has made it possible for many viewers to see video having resolutions greater than the initial recorded or received video images. For example, many television screens are capable of greater pixel resolution than the pixel resolution transmitted with standard broadcast programming or recorded on videotape or disc. Furthermore, many MPEG videos have display resolutions substantially lower than the display resolution of monitors presenting the video.
The advent of modern digital signal processing circuits and algorithms have resulted in processes for enhancing resolution of still images by processing a series of initial resolution images and determining differences there between. These differences are used to build enhanced resolution images having greater pixel density. The term “superresolution” (SR) image enhancement is used by those familiar with this area of the art to describe such image enhancement.
FIG. 1 shows a prior art method of generation of a high resolution still frame from a plurality of low resolution frames. The low resolution frame sequence 100 includes a plurality of temporal frames 122. The frames 122 are processed by super resolution frame processor 150 to produce a single high resolution frame 160. Frame 160 has enhanced resolution because the pixel density of frame 160 is substantially greater than the pixel density of each of frames 122 with a corresponding increase in displayed image resolution. For example the low resolution frame could be CIF resolution 352×288 pixels resulting in a density of 101,376 pixels per frame. The enhanced frame could be 704×576 pixels resulting in a density of 405,504 pixels per frame. Thus, image components have four time the pixel density in this example. Other embodiments may increase the high resolution image component pixel density even further and may limit the increase to only twice or less the low resolution pixel density. SR frame processor 150 analyzes relatively small differences between successive frames 120 which are due to motion of the image sensor or motion of image components in the scene. The differences are used to develop motion vectors with sub-pixel accuracy. The high resolution still frame 160 is then created by from the low resolution images and the motion vectors. While the SR frame processor does produce high resolution still frames, the process has a high computational load and is not optimally suited for generation of high definition video sequences.
FIG. 2 provides a more detailed example of the prior art system of FIG. 1 in applying superresolution still techniques to reconstruct a high resolution image from low resolution frames. The sequence of low resolution frames are shown at 222. The SR frame processor is shown at 250. The general scheme of the SR still technique is to first build up a system relating the low resolution frames to the desired high resolution image based on a image degradation model and the motion information, then utilize iteration methods to estimate the solution being the desired SR image. The following notations are used:    Lt: a low resolution at frame index (or time stamp) t.    Hk: a high resolution frame related to low resolution frame Lk.    It: an upsampled version of the low resolution frame Lt. It is generated by single frame interpolation techniques. Note that Ik serves as the initial estimate of high resolution image Hk in the iteration process to solve the SR system. Upsampled frames have the pixel density of a high resolution frame but the image resolution of a low resolution frame.    {circumflex over (M)}(t→k): the estimated subpixel motion from low resolution frames It to Ik. It is computed by utilizing It and Ik.
In this example, high resolution image Hk 260 is reconstructed by using five low resolution framesLt,tετk∪{k} whereτk={k−2, k−1, k+1, k+2} is the frame index set of the temporal related frames.While the SR frame processor does produce high resolution still frames, the process has a high computational load and is not optimally suited for generation of high resolution video sequences.
Thus, what is needed is a method and apparatus for constructing an enhanced resolution video image that efficiently builds upon SR techniques for generation of high resolution still frames. The construction should further take advantage of the high resolution video sequence itself in order to reduce the computational loading of its construction.